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Abstract 


The  photometric  sampling  method  extracts  shape  and  reflectance  properties 
of  surfaces  (Lambertian,  specular,  and  hybrid)  by  using  multiple  illumination 
directions  and  a  single  viewing  direction.  In  this  paper,  we  describe  a  complete 
implementation  and  a  performance  evaluation  of  a  three-dimensional  version  of 
the  photometric  sampler. 

All  existing  shape  extraction  techniques  that  are  based  on  photometric  mea¬ 
surements  rely  on  assumed  surface  reflection  properties.  We  proposed  a  method 
for  determining  the  shape  of  surfaces  whose  reflectance  properties  may  vary  from 
Lambertian  to  specular  without  prior  knowledge  of  the  relative  strengths  of  the 
Lambertian  and  specular  components  of  reflection. 

The  object  surface  is  illuminated  using  extended  light  sources  and  is  viewed 
from  a  single  direction.  Extended  light  sources  make  it  possible  to  ensure  the 
detection  of  both  Lambertian  and  specular  reflections.  Extended  sources  are 
generated  by  illuminating  a  diffuser  with  a  point  source  of  light.  In  the  3D 
implementation  of  the  photosampler,  multiple  extended  sources  are  generated 
by  distributing  36  point  sources  around  a  spherical  diffuser,  each  light  source  is 
located  at  the  vertex  of  a  one-frequency  icosahedron. 

An  extraction  algorithm  uses  the  set  of  image  brightness  values  measured 
at  each  surface  point  in  order  to  compute  orientation  as  well  as  the  strengths 
of  the  Lambertian  and  specular  reflection  components.  In  order  to  increase  the 
accuracy  in  measured  orientations  and  reflectance  parameters,  we  have  developed 
a  calibration  method  to  overcome  the  the  difference  between  the  theoretical  and 
the  actual  distributions  of  extended  source  brightness. 

The  experimental  conducted  on  Lambertian  surfaces,  specular  surfaces,  and 
hybrid  surfaces  show  high  accuracy  in  both  estimated  orientations  as  well  as 
reflectance  parameters.  We  have  demonstrated  the  ability  of  our  system  to  detect 
cracks,  and  reflectance  parameter  variances  in  several  industrial  objects. 


1  Introduction 


Shape  from  shading  [4, 6, 11],  photometric  stereo  [16, 5, 2],  and  local  shape  from 
specularity  [3]  are  examples  of  techniques  that  extract  three-dimensional  shape 
information  from  photometric  measurements.  All  of  these  techniques  rely  on 
prior  knowledge  of  surface  reflectance  properties.  The  reflectance  properties  are 
either  assumed,  or  measured  using  a  calibration  object  of  known  shape.  In  many 
real-world  applications,  such  as  those  involving  surfaces  of  different  reflectance 
characteristics,  the  calibration  approach  is  not  a  practical  one.  Therefore,  the 
existing  shape  extraction  methods  are  often  used  by  assuming  surface  reflectance  to 
be  either  Lambertian  or  specular.  Many  surfaces  encountered  in  practice  are  hybrid 
in  reflectance;  their  reflectance  models  are  linear  combinations  of  Lambertian  and 
specular  models.  Therefore,  Lambertian  and  specular  models  are  only  limiting 
instances  of  the  hybrid  model.  It  is  desirable  to  have  a  method  that  is  capable  of 
extracting  the  shape  of  hybrid  surfaces  as  well  as  Lambertian  and  specular  ones. 

In  many  industrial  applications,  surface  polish  and  roughness  are  found  to  be 
important  inspection  criteria.  In  such  cases,  surface  reflectance  properties  may  be 
interpreted  as  measures  of  surface  polish  and  roughness.  Furthermore,  reflectance 
properties  may  be  used  to  segment  an  image  into  different  regions;  each  region 
may  be  regarded  as  a  different  surface  to  aid  the  process  of  inspection.  For  these 
reasons,  it  would  be  of  great  value  to  have  a  technique  that  could,  in  addition 
to  determining  shape,  also  estimate  the  reflectance  properties  of  each  surface 
point  [8, 15, 7]. 

We  have  presented  the  photometric  sampling  [9]  for  determining  the  shape  of 
objects  whose  surfaces  may  be  Lambertian,  specular,  or  hybrid.  1116  method  used 
extended  light  sources  to  illuminate  an  object  surface.  A  sequence  of  images  of  the 
object  are  generated  by  actively  controlling  an  array  of  extended  sources,  which  are 
located  on  a  plane  passing  through  the  object.  Surface  orientation  was  extracted 
without  prior  knowledge  of  the  relative  strengths  of  the  Lambertian  and  specular 
reflection  components.  However,  we  had  only  implemented  and  demonstrated  2D 
version  of  the  photometric  sampling  method:  all  of  the  light  sources  were  located 
in  the  same  plane  and  only  objects  with  translated  symmetry  could  be  handled. 

This  paper  describes  the  extension  of  our  previous  work,  a  3D  version  of 
the  photometric  sampling  which  can  handle  surfaces  of  arbitray  shape.  Surface 
orientation  is  extracted  without  prior  knowledge  of  the  relative  strengths  of  the 
Lambertian  and  specular  reflection  components.  The  method  also  determines 
the  parameters  of  the  reflectance  model  at  each  surface  point  The  experimental 
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results  have  shown  high  accuracy  in  extra  ;ted  object  shape  and  surface  reflectance 
properties. 
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2  Photometric  Sampling 


2.1  Basic  Photometric  Function 

The  brightness  distribution  in  the  image  of  an  object  is  closely  related  to  the 
reflectance  properties  and  shape  of  the  object  surface,  and  the  characteristics  of 
the  light  source  used  to  illuminate  the  object.  The  basic  photometric  function 
is  defined  as  one  that  relates  image  brightness  to  the  surface  reflectance  and  the 
imaging  geometry:  surface  orientation,  viewer  and  source  direction l. 

Consider  the  illumination  of  an  object  by  a  point  source  of  light.  The  point 
source  emits  light  in  all  directions.  Light  energy  reflected  by  the  surface  in  the 
direction  of  the  camera  causes  an  image  of  the  surface  to  be  formed  in  the  camera. 
For  a  given  orientation  of  the  surface  and  direction  of  the  point  source,  the  amount 
of  light  energy  reflected  by  the  surface  in  a  particular  direction  is  determined 
by  its  reflectance  properties.  The  photometric  function  of  a  large  number  of 
surfaces  comprises  three  components,  namely,  the  Lambertian  (diffuse),  the  lobe 
(rough)  component,  and  the  specular  (gloss)  component  [10].  Figure  1  shows  the 
relationship  between  these  three  components  and  the  imaging  geometry. 

Our  photometric  sampling  method  handles  smooth  surfaces  2.  Then,  we  can 
neglect  the  specular  lobe  component  and  express  the  brightness  at  an  image  point 
as: 


7  =  IL  + IS.  (1) 

where  IL  and  IS  denotes  the  image  brightness  due  to  the  Lambertian  component 
and  the  specular  spike  component,  respectively. 

The  Lambertian  brightness  IL  results  from  the  non-homogeneous  nature  of  the 
layers  that  constitute  the  object  surface.  Light  rays  penetrating  the  surface  undergo 
multiple  refractions  and  reflections  at  the  boundaries  between  surface  layers  and 
reemerge  near  the  point  of  entry  with  a  variety  of  directions.  These  surfaces  appear 
equally  bright  from  all  directions;  the  brightness  IL  is  independent  of  the  viewing 
direction. 

'The  photometric  function  is  similar  to  the  image  irradiance  equation  [4];  both  represent  the 
relationship  between  the  image  brightness  and  the  imaging  geometry.  The  reader  may,  therefore, 
regard  the  photometric  function  as  an  unnormalized  image  irradiance  equation. 

2 More  precisely,  we  assume  that  the  surface  roughness  measure  g  is  much  smaller  than  1. 
See  [10]. 
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Figure  1:  Polar  plots  of  the  three  reflection  components  as  the  function  of  the 
viewing  angle  for  a  fixed  source  direction 
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The  brightness  of  a  Lambertian  surface  point  is  proportional  to  the  energy  of 

received  incident  light.  The  amount  of  light  energy  falling  on  the  surface  element 

is  proportional  to  the  area  of  the  surface  element  as  seen  from  the  point  source 

position,  often  referred  to  as  the  foreshortened  area.  The  foreshortened  area  is  a 

cosine  function  of  the  angle  0  between  the  surface  orientation  direction  N  and  the 
^  ■*  ^ 

source  direction  S  from  the  surface  point,  where  N  and  S  are  unit  vectors.  Thus, 
the  Lambertian  brightness  component  IL  may  be  written  using  these  vectors  as: 

IL  =  A'cosp  (2) 

=  AS  ■  N 

where  A  is  a  constant  representing  ratio  of  absorption  between  the  incoming  energy 
and  emitting  energy,  the  Lambertian  albedo. 

The  specular  spike  brightness  IS  results  from  the  light  rays  that  are  unable  to 
penetrate  the  surface  layer  and  reflected  back  to  the  air.  It  is  represented  by  a  very 
sharp  function  of  the  source  direction  S  with  respect  to  the  viewer  and  the  surface 
orientation.  We  can  approximate  it  by  a  unit  impulse  function: 

IS  =  BU(N,S)  (3) 

where: 

U(N,S)  =  (  1 

[  0  otherwise 

and  B  is  the  specular  spike  and  Vis  a  unit  normal  representing  the  viewer  direction. 

The  photometric  sampling  method  determines  orientation  and  reflectance  un¬ 
der  a  fixed  viewer  direction.  We  can  set  up  the  viewer  center  coordinate  system  so 
that  this  viewer  direction  coincides  to  the  Z  axis;  V  is  denoted  as  (0, 0, 1)'.  We  can 
represent  source  and  surface  orientation  directions  with  respect  to  this  coordinate 
system. 

By  moving  the  source  directions  around  the  object,  we  can  vary  the  source 
direction  without  changing  the  surface  orientation  and  reflectance  parameters. 

It  is  important  to  note  that  varying  the  source  direction  does  not  change  the 
orientation  and  reflectance  parameters.  Therefore,  even  though  the  orientation  and 
reflectance  parameters  are  unknown,  we  can  treat  them  as  constants  in  equation  1. 
For  this  reason,  we  will  often  refer  to  the  basic  photometric  function  as  7(5),  a 
relation  between  image  brightness  and  source  direction. 
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We  will  denote  the  surface  orientation  direction  which  satisfies  N  =  4^  for 
a  given  source  direction  as  the  specular  surface  direction  of  the  source  direction, 
N,(S).  The  source  direction  which  satisfies  this  equation  for  a  given  surface 
direction  as  the  specular  source  direction  of  the  surface,  S„(N). 

Figure  2  shows  plots  of  the  basic  photometric  function  as  the  function  of  the 
moving  light  source  in  the  viewer  centered  coordinate  system. 

2.2  Extended  Light  Source 

Numerous  point  source  illumination  methods  [2,  12,  16,  14],  have  been  proposed 
that  extract  the  shape  of  either  Lambertian  or  specular  surfaces.  A  problem 
associated  with  the  use  of  point  sources  is  that  the  specular  component  of  reflection 
is  not  detected  unless  the  IV  =  Ns(S).  To  detect  specular  reflections  from  surface 
points  of  all  orientations,  an  infinite  number  of  point  sources  need  to  be  positioned 
around  the  surface.  Such  an  approach  is  not  practical. 

We  will  use  extended  light  sources  to  illuminate  the  object  surface  [5].  Unlike 
a  point  source,  an  extended  source  emits  light  from  an  area  of  points  rather  than 
a  single  point.  This  characteristic  of  an  extended  source  reduces  the  number  of 
necessary  sources  to  detect  specular  components  of  reflection. 

In  order  to  determine  shape  and  reflectance  parameters  of  hybrid  surfaces,  both 
specular  and  Lambertian  reflections  must  be  captured  in  the  measured  brightness. 
However,  experiments  have  shown  that  for  point  source  illumination,  it  is  difficult 
to  measure  both  Lambertian  and  specular  reflections  without  changing  the  sensi¬ 
tivity  of  the  measuring  device.  Image  brightness  due  to  specular  reflections  have 
been  observed  to  be  much  higher  than  the  brightness  resulting  from  Lambertian 
reflections. 

For  extended  source  illumination,  a  specular  surface  element  of  a  given  orien¬ 
tation  will  reflect  light  from  a  small  area  of  the  extended  source  into  the  camera. 
On  the  other  hand,  a  Lambertian  surface  element  of  the  same  orientation  reflects 
light  from  all  points  of  the  extended  source.  Due  to  this  integration  effect,  under 
an  extended  source,  the  the  relative  brightness  of  a  Lambertian  component  to  those 
specular  component  under  an  extended  source  is  much  greater  than  under  a  point 
source  due  to  the  integration  effect  Therefore,  an  extended  light  source  makes  it 
possible  to  detect  both  Lambertian  and  specular  reflections  without  changing  the 
sensitivity  of  the  sensor. 

We  generate  an  extended  source  by  illuminating  a  sphere  made  of  light- 
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moving  light  source. 


9 


diffusing  material  with  a  point  source  of  light.  The  object  is  placed  at  the  center  of 
the  spherical  diffuser  and  is  viewed  by  the  camera  through  a  hole  in  the  diffuser. 
In  the  following  discussion,  the  origin  of  the  viewer  center  coordinate  system  is 
defined  as  the  sphere  center,  and  the  Z  axis  corresponds  to  the  optical  axis  of  the 
TV  camera.  All  vectors,  such  as  S  and  N,  are  defined  in  this  viewer  centered 
coordinate  system. 

Each  point  source  illuminates  the  spherical  diffuser,  and  the  illuminated  area 
on  the  spherical  diffuser  acts  as  an  extended  light  source.  Brightness  at  any 
point  on  the  sphere,  P,  is  described  by  the  brightness  distribution  function  of  the 
extended  light  source,  L(P,  S),  given  by  the  illumination  geometry.  The  brightness 
distribution  function  has  the  following  characteristics: 

•  the  position  of  an  extended  source  is  defined  as  the  direction,  S,  of  the  point 
source  which  generates  the  extended  source, 

•  the  brightness  distribution  function  has  a  non  zero  area.  It  is  bounded  by 
a  circle  whose  radius  is  a  and  whose  center  is  the  position  of  the  extended 
source.  In  other  words,  L  =  0  for  >  a,  where  /?  *  cos~1(S  ■  P).  We  will 
refeT  to  a  as  the  termination  angle  of  the  extended  light  source, 

•  the  brightness  distribution  function  is  rotational  symmetric  with  respect  to 
the  source  position  S, 

•  its  magnitude  decreases  as  P  deviates  from  S. 

2.3  Photometric  Function  for  Extended  Sources 

We  will  modify  the  point  source  illumination  photometric  function  (Eq.  1)  into 
an  extended  source  illumination  photometric  function.  An  extended  source  may 
be  thought  of  as  a  collection  of  point  sources.  Each  point  source  has  a  brightness 
that  is  dependent  on  its  position  in  the  extended  source.  The  brightness  of  light 
reflected  by  an  object  surface  that  is  illuminated  by  an  extended  source  may  be 
determined  by  computing  the  integral  of  the  light  energy  reflected  from  all  points 
on  the  extended  source.  Therefore,  the  modified  photometric  function  I'(S)  is 
determined  by  convolvingjhe  basic  photometric  function  I(S)  with  the  brightness 
distribution  function  L(JP,  S). 
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For  a  surface  point  of  orientation  N  the  Lambertian  component  IV  of  the 
modified  photometric  function  is  determined  as: 

IV  =  J  J  IL(P)L(P,  S)dP 
=  J  J  A(N  ■  P)  L(P,  S)  dP 

=  A'N-S  (4) 

The  specular  component  of  the  modified  photometric  function  is  proportional 
to  the  brightness  of  the  extended  source  at  the  specular  source  direction,  Sn(N). 
Therefore,  the  photometric  function  for  surface  illumination  by  the  extended  source 
is: 

I'  =A'N  •S  +  B'  L(Sn(N),  S)  (5) 

Eq.5  is  the  modified  photometric  function  that  relates  image  brightness  to 
surface  orientation,  surface  reflectance,  and  source  position  for  extended  source 
illumination.  In  the  equation,  surface  orientation  and  reflectance  parameters  are 
unknown,  but  constant,  for  a  given  surface  point.  For  this  reason,  we  will  fre¬ 
quently  refer  to  the  modified  photometric  function  as  a  relation  between  image 
brightness  and  extended  source  direction,  /'(S). 

2.4  Sampling 

The  process  of  measuring  image  brightness  for  different  source  directions  is  equiv¬ 
alent  to  sampling  the  modified  photometric  function,  /'(S),  at  various  S,.  Samples 
of  the  photometric  function  may  be  obtained  by  moving  an  extended  source  around 
the  object  and  taking  images  of  the  object  for  different  source  positions.  An  al¬ 
ternative  approach  would  be  to  distribute  an  array  of  extended  sources  around  the 
object  such  that  each  source  illuminates  the  object  from  a  different  direction.  The 
entire  array  of  extended  sources  may  be  scanned  by  sequentially  activating  each 
source  one  at  a  time  and  taking  an  image.  Therefore,  the  scanning  process  results 
in  a  set  of  brightness,  /,  :  i  =  1  ,...,M,  measured  at  each  point  on  the  object 
surface.  M  equals  the  number  of  extended  sources. 

The  number  of  samples  measured  at  each  surface  point  is  determined  by  the 
termination  angle  a.  As  stated  earlier,  in  order  to  extract  the  shape  and  reflectance 
parameters  of  hybrid  surfaces,  both  Lambertian  and  specular  components  of  image 
brightness  must  be  detected.  Since  we  have  used  a  unit  impulse  specular  reflection 
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model,  the  area  of  the  photometric  function  that  contains  specular  brightness  is 
bounded  by  an  a  radius  circle. 

Uniform  sampling  of  the  photometric  function  may  be  obtained  by  distributing 
extended  sources  in  a  tessellation  that  results  from  projecting  onto  the  unit  sphere 
a  regular  polyhedron  whose  center  coincides  with  that  of  the  sphere[  1].  A  higher 
frequency  polyhedron  which  consists  of  triangular  faces  can  be  used  [13]  for  a 
smaller  meshed  tessellation. 

We  will  use  a  polyhedron  whose  edge  length  projects  to  an  arc  length  equivalent 
to  a.  This  spacing  allows  the  photometric  function  to  be  uniformaly  sampled  at 
an  interval  equal  to  a.  Under  this  sampling  interval,  one  and  only  one  spherical 
triangle  gives  three  non-zero  specular  brightnesses.  (See  Figure  3.)  We  will  use 
this  constraint  for  designing  a  recovery  algorithm.  We  will  refer  to  this  triangle  and 
the  constrain  as  the  specular  triangle  and  the  sampling  constraint ,  respectively. 

2.5  Extracting  Shape  and  Reflectance  of  Surfaces 

Given  the  set  of  image  brightness  values,  /, :  i  =  1 , . . . ,  M  we  want  to  determine  the 
surface  orientation,  N,  and  the  strengths.  A'  and  B\  of  the  Lambertian  and  specular 
components  of  reflection.  The  shape  and  reflectance  properties  extracted  at  each 
surface  point  are  solely  based  on  the  image  brightness  recorded  at  that  point.  The 
surface  property  values  assigned  to  a  surface  point  are  not  influenced  by  the  image 
brightness  measured  at  neighboring  points.  We  will  first  develop  techniques  to 
extract  surface  orientations  of  purely  Lambertian  and  purely  specular  surfaces. 
Later,  these  techniques  will  be  used  as  tools  to  extract  surface  orientations  and 
reflectance  properties  of  hybrid  surfaces. 

Lambertian  Surfaces  Consider  the  case  where  the  surface  of  an  object  is  known 
to  be  purely  Lambertian,  and  the  shape  of  the  object  is  to  be  determined.  The 
photometric  samples  for  a  Lambertian  surface  may  be  written  as: 

/,  =  A'  N  ■  Si  (6) 

We  would  like  to  compute  the  surface  orientation,  N,  and  and  the  strength, 
A',  of  the  Lambertian  reflection  component.  These  variables  may  be  ejtimated  by 
fitting  the  equation  to  the  set  of  image  brightness,  /»  :  /  =  1, . . . ,  M  and  extended 
source  directions,  5; :  /  =  1, . . . ,  Af,  using  the  least  square  fitting  method. 
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Figure  3:  Sampling  constraint:  one  and  only  one  spherical  triangle  produces  all 
three  non-zero  specular  brightness  values. 
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Specular  Surfaces  Now  consider  the  case  where  the  object  surface  is  known 
to  be  purely  specular,  and  the  shape  of  the  object  is  to  be  determined.  From  the 
sampling  constraint,  we  can  assume  that  only  three  brightness  values  are  non-zero, 
and  the  corresponding  three  directions  form  a  specular  triangle. 

Given  the  image  brightness  set,  /,  :  i  =  1, . .. ,  Af,  at  a  surface  point,  the  three 
non-zero  image  brightness  values  in  the  set  are  first  determined.  These  three 
values,  //,/«,/*,  and  their  corresponding  source  directions,  must  satisfy 

the  following  four  equations: 

h  =  B'USMJi) 

Im  =  B'L(S„(N)X )  (7) 

In  =  B'L(Sn(N)X) 

\N\  =  1 

By  solving  these  four  equations,  we  can  determine  the  surface  orientation,  N, 
and  specular  strength,  B'  at  each  point.  In  the  actual  implementation,  however, 
we  use  a  lookup  table  to  convert  a  triple  of  brightness  values  into  a  unique  surface 
orientation  [5]. 

Hybrid  Surfaces  A  large  number  of  surfaces  may  be  classified  as  hybrid  surfaces 
whose  photometric  functions  are  linear  combinations  of  Lambertian  and  specular 
components.  The  hybrid  surface  class  includes  plastic  surfaces  and  painted  smooth 
surfaces.  We  will  develop  an  algorithm  to  recover  the  Lambertian  strength  and 
specular  strength,  and  surface  orientation  of  a  hybrid  surface.  The  extraction 
algorithm  is  developed  based  on  the  sampling  constraint. 

This  method  assumes  that  all  the  image  brightness  does  not  contain  any  spec¬ 
ular  component  We  will  determine  the  surface  orientation,  Mam*,  and  the  Lam¬ 
bertian  strength,  A ',  using  the  least  square  fitting  method  as  mentioned  above. 

Then,  we  will  calculate^ the  theoretical  brightness,  1\  :  i  =  1, ... ,  Af,  for  all 
source  directions  using  A',  Niamb,  and  5,  :  /  =  1 , . . . ,  M.  By  subtracting  theoretical 
brightness  values  from  the  observed  ones,  we  obtain  residual  brightness  values 
Ali  =  I'i—Ii.  The  residual  brightness  represents  specular  brightness.  The  sampling 
constraint  guarantees  that  only  three  residual  values  contain  non-zero  values.  By 
finding  these  residual  brightness,  we  can  identify  the  specular  triangle.  Using  the 
three  brightness  residual  brightness  and  the  specular  triangle,  we  can  determine 
the  surface  orientation,  Nspee,  and  the  specular  strength,  B\  by  using  the  same 
procedure  as  was  used  for  specular  surfaces. 
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Third,  we  will  use  the  least  square  fitting  method  on  the  remaining  bright¬ 
ness  values  and  redetermine  the  surface  orientation,  Nuimb,  and  the  Lambertian 
strength.  A'.  Using  these  values,  we  will  recalculate  the  brightness  residues.  We 
find  the  three  largest  residues  and  redetermine  the  specular  triangle.  Then,  we 
again  estimate  the  specular  strength,  B\  and  surface  orientation,  Nspec,  from  these 
residues. 

We  have  two  surface  orientations  given  by  two  different  methods  (Lambertian 
and  specular).  We  compute  a  weighted  average  of  both  surface  orientations  into  a 
single  value  based  on  the  ratio  of  Lambertian  and  specular  strength: 


3  Three-Dimensional  Photo-sampler  Apparatus 


3.1  Designing  the  Apparatus 


We  produce  an  extended  source  by  illuminating  a  spherical  diffuser  with  a  point 
source.  The  object  is  placed  at  the  center  of  the  spherical  diffuser.  Point  light 
sources  are  distributed  uniformly  over  the  diffuse  sphere.  Each  point  source 
illuminates  a  part  of  the  diffuser,  and  that  part  acts  as  an  extended  source.  This 
illuminated  area  on  the  diffuse  sphere  is  referred  to  as  the  effective  area  of  the 
extended  source.  The  effective  area  is  bounded  by  a  spherical  circle  whose  radius 
is  the  termination  angle  a  and  whose  center  is  the  light  source  direction. 

The  termination  angle  determines  the  number  of  point  sources  necessary  to 
cover  the  entire  surface  of  spherical  diffuser.  In  terms  of  efficiency,  we  prefer 
a  larger  a  value.  A  large  a  needs  fewer  sources  to  cover  the  spherical  surface. 
But,  a  large  a  degrades  the  resolution  of  the  surface  orientation  measurements, 
because  the  gradient  of  extended  sources  become  flat.  A  smaller  a  gives  a  better 
surface  orientation  resolution  because  of  the  steep  gradient  of  extented  sources. 
On  the  other  hand,  the  area  of  the  extended  source  becomes  smaller.  We  would 
require  more  sources.  A  smaller  a  also  makes  the  Lambertian  surfaces  relatively 
darker  than  the  specular  surfaces.  This  makes  it  difficult  to  extract  the  Lambertian 
component.  From  several  experiment,  we  select  40°  as  the  optimal  termination 
angle:  a  =  40°.  This  configuration  of  point  light  source  distribution  can  be 
realized  by  locating  point  sources  at  each  vertex  of  one  frequency  icosahedron. 
This  requires  42  light  sources. 

The  distance  H  between  a  point  source  and  the  surface  of  the  diffuser  is  related 


to  the  termination  angle  a: 


cosa  = 


R 

R+H' 


(8) 


where  R  denotes  the  radius  of  the  diffuser.  See  Figure  4.  Therefore,  once  we 
determine  the  termination  angle,  a,  we  can  determine  H  as  H  = 

By  using,  R  and  H  we  can  express  the  brightness  distribution  function  of  an 
extended  light  source  as 


L(P,S;R,H) 


_ C[(R+H)(SP)-R] _ 

[(R  +  H  -  R  (S  •  P))2  +  (fl(l  -  (S  •  P)2)]l 


(9) 


where  P  denotes  a  unit  vector  to  a  point  on  the  diffuse  surface  from  the  sphere 
center  and  S  denotes  the  point  light  source  direction  from  the  sphere  center. 
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Figure  4:  The  relationship  between  a  and  H 


Figure  5  shows  a  photograph  of  the  experimental  device.  A  355.6mm  (14- 
inch)  diameter  lamp  shade  is  used  as  the  spherical  diffuser.  Extended  light  sources 
are  generated  on  the  diffuser’s  surface  by  illuminating  it  with  36  60W  frosty 
incandescent  light  bulbs.  The  bulbs  are  located  at  36  of  the  42  vertices  of  one- 
frequency  icosahedron 3.  The  object  is  placed  at  the  center  of  the  diffuser  and  is 
viewed  by  a  camera  through  a  25.4mm  (1-inch)  diameter  hole  in  the  surface  of 
the  diffuser.  The  complete  imaging  system,  comprise  of  lenses  and  camera,  has  a 
physical  resolution  of  0.5mm  (0.002  inches)  per  pixel  width. 

3.2  System  Parameters 

The  system  parameters  needed  to  describe  the  apparatus  in  details  include:  light 
source  directions,  brightness  distribution  parameters,  and  device  characteristic 
parameters.  These  parameters  will  be  used  for  determining  shape  and  reflectance 
properties  of  unknown  objects. 

3  A  small  section  of  the  bottom  hemisphere  of  the  diffuser  is  not  illuminated  as  light  sources. 
This  area  is  not  observable  from  the  TV  camera. 
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Figure  5:  Photograph  of  the  experimental  apparatus. 


•  The  light  source  directions  represent  directions  of  the  center  of  the  light 
sources  denoted  by  Si  :  i  *  1, . . . ,  36.  Figure  6  shows  the  number  of  the 
light  sources  and  their  locations. 

•  The  brightness  distribution  parameters ,  C,H,D ,  and  <r,  characterize  the 
shape  of  the  brightness  distribution  of  a  light  source  on  the  diffuser.  Here 
the  function  is  expressed  by 


L'{P,S',R,H)  = 


C[(R  +  H)($P)-R] 


[(R  +  H-R  (5  •  P))2  +  RH1  -  (5  •  F)2)]i 

(cos-HSP))\ 


+D  exp(- 


2a1 


•) 


(10) 
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For  an  ideal  diffuser,  the  first  term  in  the  above  the  equation  is  accurate. 
However,  since  the  sphere  is  not  an  ideal  Lambertian  diffuser,  we  add  a 
Gaussian  function  to  compensate  this  discrepancies.  C  and  D  represent  the 
strength  of  the  Lambertian  and  Gaussian  functions,  respectively.  H  and 
a  are  related  with  the  width  of  the  Lambertian  and  Gaussian  functions, 
respectively. 

•  Device  characteristic  parameters 

-  The  base  value  for  specular  surfaces  ( Esptc )  represents  the  increase  in 
brightness  due  to  interreflections  inside  the  diffuser.  We  assume  that 
this  increment  is  independent  on  surface  orientations. 

-  The  base  value  for  Lambertian  surfaces  ( Eiamb )  represents  the  increase 
in  brightness  on  Lambertian  surface  due  to  interreflections  inside  of 
the  diffuser.  We  assume  that  this  increment  is  also  independent  on 
surface  orientations.  This  factor  corresponds  to  the  integration  of  the 
previous  factor  over  the  hemispherical  directions. 

-  The  light  source  brightness  for  the  pure  Lambertian  surface  ( A'^  ) 
represents  the  brightness  that  an  effective  point  source  would  have,  in 
order  to  produce  the  same  brightness  as  an  extended  source.  Theo¬ 
retically,  this  value  corresponds  to  the  integration  of  Eq.10  over  the 
effective  illuminated  area  of  an  extended  source.  We  determine  this 
value  by  calibration  for  the  sake  of  accuracy. 

-  The  base  value  due  to  defocus  effect  (£<*,/)  represents  the  increase  in 
brightness  over  the  entire  image  because  the  camera  observes  a  part  of 
the  diffuser  and  blurs  that  part  over  the  image. 


3.3  Calibration 

We  determine  the  system  parameters  and  lookup  tables  by  a  calibration  method. 
The  system  parameters,  except  for  £,*,/ ,  are  determined  for  each  extended  light 
source.  The  lookup  tables  are  determined  for  each  triangle  on  the  one-frequency 
icosahedron. 

This  calibration  method  takes  a  series  of  images  of  a  specular  sphere  and  a 
Lambertian  sphere,  and  determine  the  parameters  and  tables.  The  light  source 
directions,  brightness  distribution  parameters,  base  value  for  specular  surfaces. 
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Figure  6:  Map  of  the  distribution  of  light  sources 


and  the  base  value  for  the  defocus  effect  are  determined  by  processing  36  images 
of  a  specular  sphere  under  the  36  extended  light  sources.  The  lookup  tables  are  also 
constructed  using  36  images  of  the  specular  sphere.  The  light  source  brightness 
for  the  pure  Lambertian  surface  and  base  value  for  the  Lambertian  surfaces  are 
determined  by  processing  36  images  of  a  Lambertian  sphere  under  the  36  extended 
light  sources. 

3  J.l  Device  characteristic  parameters 

Light  source  directions  We  will  repeat  the  following  procedure  for  all  36  light 
sources.  First,  we  turn  on  a  light  source  and  take  an  image  of  the  specular  sphere. 
Then,  we  get  the  maximum  brightness  around  the  image  point  estimated  as  the  light 
source  direction  by  the  specification  of  the  device.  Third,  we  make  a  binary  image 
using  a  threshold  which  equals  90  %  of  the  maximum  brightness,  and  swell  the 
image  by  3  pixels  to  avoid  the  influence  of  defects,  stain,  and  dust.  The  direction 
of  the  light  source  is  estimated  as  the  center  of  the  binary  region.  This  estimation 
will  be  used  as  the  initial  value  for  the  more  precise  calibration.  Figure  7  shows 
the  distribution  of  such  binary  areas  extracted  on  the  specular  ball  plotted  on  a 
plane. 

Base  values  for  specular  surfaces  Theoretically,  the  brightness  distribution 
function,  Eq.10,  should  give  zero  brightness  outside  of  the  effective  area  of  an 
extended  source.  However,  interreflections  inside  the  diffuser  illuminate  the  whole 
diffuser.  This  produces  a  certain  brightness  in  areas  outside  of  the  effective  area. 
(See  Figure  8.)  The  interreflections  depend  on  the  shape  and  material  of  the 
diffuser,  and  the  direction  of  the  extended  source.  Since  this  brightness  is  about  1 
%  of  the  maximum  value  of  the  extended  light  sources,  we  neglect  the  differences 
between  the  light  sources  and  assume  it  a  constant  over  all  the  directions.  In  other 
words,  we  assume  that  we  will  observe  a  uniform  increment  in  brightness  over  an 
image. 

We  obtain  this  base  value  from  the  difference  in  brightness  of  a  point  on  the 
camera  hole  image  and  a  point  under  the  diffuser  on  the  specular  ball.  Note  that 
we  should  use  a  light  source  far  from  the  camera  in  order  to  avoid  the  defocus 
effect  stated  later. 
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Figure  7:  Binary  area  of  light  sources 
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Figure  8:  Interreflection  in  the  diffuser 

Base  values  for  the  defocus  effect  Because  of  the  insufficient  size  of  the  camera 
hole  on  the  diffuser,  defocused  parts  of  the  diffuser  get  into  the  view  of  the  camera  4. 
This  causes  an  increase  in  the  brightness  of  the  whole  images. 

The  images  under  light  sources  near  the  camera  have  this  effect  much  stronger 
than  those  under  the  far  sources.  Thus,  we  measure  the  defocus  effect  at  each 
light  source  independently.  We  take  a  specular  ball  image  under  a  light  source. 
Brightness  corresponding  to  the  outside  of  the  effective  area  of  the  light  source 
has  non-zero  brightness.  This  brightness  is  caused  by  interreflection  and  defocus. 
In  order  to  measure  the  non-zero  outside  brightness,  we  take  an  average  of  the  two 
darkest  values,  among  four  orthogonal  brightness  values,  along  a  large  circle  (in 
our  case,  0.85%  of  the  radius  of  the  specular  sphere  image).  We  assume  that  the 
two  darkest  values  are  from  outside  zero-brightness.  Then,  we  subtract  the  base 
value  for  specular  surfaces  (interreflection),  determined  in  the  previous  step,  from 
this  value  to  get  those  due  to  the  defocus  effect. 

‘Reader  may  consider  if  wc  use  a  wider  hole  we  can  avoid  this  effect  However,  a  wider  hole 
increase  a  hole  region  in  specular  image,  and  should  be  avoide. 
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Figure  9:  Sampling  the  intensities  on  the  diffuser 
3.3.2  Brightness  distribution  parameters 

The  brightness  distribution  parameters  C,H,D  ,cr  characterize  the  shape  of  the 
brightness  distribution  on  the  diffuser.  By  sampling  the  real  brightness  distribution 
on  the  diffuser  and  fitting  them  to  Eq.  10,  we  will  determine  these  parameters. 
Since  these  parameters  are  unique  to  each  light  source,  we  will  repeat  the  fo' lowing 
procedure  at  each  light  source.  For  this  calibration,  we  use  36  images  of  a  specular 
ball  under  36  light  sources. 

We  will  sample  real  brightness  at  each  3.6°  interval  within  36°  limits  along 
the  longitudinal  and  the  perpendicular  great  circle  intersecting  at  the  light  source 
direction.  At  each  sampling  angle,  we  search  the  maximum  brightness  within  20° 
along  the  arc  perpendicular  to  the  sampling  circle  to  minimize  the  effect  of  defects, 
stain  and  dust  on  the  sampling  sphere.  (See  Figure  9.) 

By  fitting  the  equation  to  the  two  sets,  longitudinal  and  perpendicular,  of 
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sampled  brightness,  two  sets  of  parameters  C,H,D,a,  and  89  are  determined. 
Then,  these  two  sets  of  parameters  are  averaged.  Here,  C,  H,D,  <j  are  the  same  as 
those  in  Eq.  10  and  86  denotes  the  difference  between  the  real  and  the  estimated 
light  source  direction. 

Each  fitting  process  is  executed  in  the  following  three  steps.  The  first  step 
determines  the  parameters  for  the  Lambertian  component,  86,  C,  H,  and  the  second 
step  determines  those  for  the  Gaussian  component,  D  and  a.  The  last  step  adjusts 
the  relative  strength,  C  and  D  so  that  the  difference  between  the  real  one  and  the 
one  described  by  these  parameters  is  minimized. 

The  first  step  approximates  the  brightness  distribution  by  the  Lambertian  com¬ 
ponents  of  Eq.  10  and  determines  the  parameters,  66,  C  and//by  using  a  non-linear 
least  square  fitting  method.  Using  H  given  by  the  device  dimension  and  D  given 
by  the  maximum  brightness  value,  the  method  obtains  the  86  which  gives  the  min¬ 
imum  error  among  the  possible  86  values,  ±5°  limits  around  the  estimated  light 
source  direction  and  0.1°  intervals.  Then,  the  method  determines  the  height  H  that 
gives  the  minimum  error  under  the  C  given  by  th,  maximum  brightness.  Finally, 
using  these  H  and  C  as  the  initial  values,  the  method  determines  the  optimal  H  and 
C  for  the  Lambertian  component  of  Eq.  10  by  the  Gauss-Newton  non-linear  least 
square  fitting  method,  iteratively. 

The  second  step  approximates  the  brightness  distribution  by  the  Gaussian 
component  of  Eq.  10  and  determines  the  parameters,  D  and  a  of  Eq.  10.  D  and  cr 
are  determined  by  the  Gauss-Newton  non-linear  least  square  fitting  method.  The 
initial  value  of  D  is  given  by  the  maximum  brightness  value.  The  initial  value  of 
a  is  chosen  so  that  the  Gaussian  component  gives  the  same  value  as  the  one  given 
by  the  Lambertian  component  at  9°  =  0.1571  radian.  Namely,  a  =  — j5— ,  where 

V 

Lo  denotes  the  brightness  at  9°  from  the  light  source  direction. 

The  third  step  adjusts  the  ratio  between  the  Lambertian  and  Gaussian  com¬ 
ponents.  Fixing  all  parameters  except  C  and  D,  we  obtain  these  two  parameters 
by  using  the  least  square  fitting  method.  Figure  10  shows  some  of  the  brightness 
distribution  function  superimposed  on  the  measured  brightness. 

3.33  Lookup  table  for  surface  orientations  given  specular  component 

A  specular  surface  obeys  the  law  of  reflection  and  gives  the  brightness  of  an 
extended  source  at  the  specular  source  direction.  From  observing  a  triple  of  these 
brightness  given  by  three  different  extended  sources,  we  can  determine  the  surface 
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orientation  of  the  surface  as  described  in  section  2.5. 

The  device  consists  of  many  triangular  tessellations.  Within  one  triangular 
tessellation,  the  effective  areas  of  three  extended  light  sources  overlap  with  each 
other.  We  will  make  the  lookup  table  at  each  triangular  tessellations. 

We  will  construct  a  lookup  table  to  convert  a  triple  of  brightness  values  to 
a  surface  orientation  analytically.  Here  the  axes  of  the  three  dimensional  table 
correspond  to  three  brightness  values,  and  each  cell  of  the  table  contains  a  surface 
orientation  corresponding  to  a  triple. 

We  sample  orientation  over  a  triangular  tessellation.  At  each  sampled  orienta¬ 
tion,  we  can  calculate  the  three  brightness  values  using  Eq.  10  analytically.  Here 
the  parameters  for  the  equation  are  given  by  the  calibration  method  in  the  previous 
section.  Then,  we  place  the  corresponding  surface  orientation  at  the  cell  of  the 
table,  corresponding  to  the  calculated  triple  of  the  brightness.  In  order  to  avoid 
null  spaces  in  the  lookup  table  due  to  the  insufficient  sampling,  we  interpolate  and 
extrapolate  the  contents  of  the  table. 

3  J.4  Light  source  brightness 

The  light  source  brightness  for  the  pure  Lambertian  surface,  A'^,  and  the  base 
value  for  the  Lambertian  surface.  Eiam,  are  obtained  by  calibration  at  each  light 
source.  Here,  the  brightness  value  of  a  pure  Lambertian  surface  can  be  described 
as: 

A'^aS-M  +  Eun)  (11) 

where  5  and  N  denote  the  light  source  direction  Both  the  base  value  and  the  source 
brightness  can  be  obtained  by  integrating  appropriate  equations.  However,  we 
will  determine  these  parameters  experimentally  for  the  sake  of  accuracy. 

These  parameters  are  extracted  by  processing  the  36  images  of  a  pure  Lam¬ 
bertian  ball  under  36  light  sources  independently.  At  each  image,  we  sample,  at 
12°,  a  set  of  brightness  values  along  the  longitudinal  great  circle  of  the  Lambertian 
sphere  which  passes  through  the  light  source  direction.  We  can  calculate  N  ■  S  at 
each  sampling  point  Substituting  these  values  and  brightness  values  to  Eq.  11, 
we  can  determine  A'^  and  Eum  by  the  least  square  fitting  method. 

When  a  surface  normal  makes  an  angle  near  ±90°  with  respect  to  the  light 
source  direction,  part  of  the  extended  light  source  is  occluded  by  the  surface  itself. 
The  brightness  departs  from  the  cosine  curve.  Therefore,  only  the  brightness 
values  given  by  the  normal  direction  between  ±60°  are  used  for  this  fitting. 
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The  camera  cannot  see  the  entire  area  illuminated  by  an  oblique  direction  of 
a  source,  due  to  occlusion  by  the  Lambertian  sphere.  We  can  observe  brightness 
only  at  the  peripheral  part  of  the  cosine  curve.  In  this  case,  the  fitting  method 
becomes  unstable.  Thus,  we  artificially  generate  a  symmetric  set  of  brightness 
with  respect  to  the  light  source,  and  apply  the  least  square  fitting  to  the  combined 
set  of  measured  and  the  artificially  generated  brightness  values. 

3.4  Extraction  Procedure 

Image  brightness  at  each  pixel  under  the  extended  light  source  i  can  be  represented 
using  the  system  parameters  as: 

Ii(N,  A’,B')  =  A' (1  ■  N  +  Euunb)  +  B'&'il,  N)  +  Espec)  +  (12) 

Here,  the  device  characteristic  parameters,  Eipec,  E^f ,  the  light  source  di¬ 
rection  Si,  and  the  light  source  brightness  for  the  pure  Lambertian  surface,  A'^ 
are  given  by  the  previous  calibration  procedure.  The  number  of  the  light  sources 
is  36.  Namely,  i*  =  1,  — ,  36^ 

We  will  recover  A',  B',  N  from  step  1  through  step  3  several  times  iteratively 
to  obtain  the  accurate  parameters.  Before  entering  the  main  part  of  the  iteration, 
we  average  the  brightness  values  by  using  the  following  filter. 

1  1  1 
1  8  1 
1  1  1 

Step  1:  Cosine  curve  fitting  A  measured  brightness  value,  /,  includes  the  three 
kinds  of  base  values:  Lambertian  (Eumb),  specular  (Ev*e),  and  defocus  (Ed*/),  we 
subtract  these  base  values  from  the  brightness. 

I; =  EfambA  EspecB  Ed*/ . 

In  the  initial  iteration.  A1  and  B'  are  unknown.  Therefore,  we  set  A'  -  0  and 
B?  s=0.  After  second  iteration,  we  will  substitute  the  values  obtained  at  the 
previous  iteration. 

Then,  the  brightness  is  normalized  by  the  the  light  source  brightness  for  the  pure 
Lambertian  surface,  AL^  .,  of  source  i  to  compensate  for  the  difference  between 
the  light  sources. 
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Brightness  values  larger  than  a  threshold  (0.05)  are  considered  as  valid  mea¬ 
surements  and  are  used  for  fitting.  This  is  because  the  actual  photometric  function 
of  Lambertian  surfaces  does  not  follow  a  cosine  curve  in  the  peripheral  area.  If 
the  number  of  valid  measurements  is  larger  than  a  threshold  (currently,  seven 
including  the  light  sources  on  the  specular  triangle),  the  results  of  the  fitting  are 
regarded  as  valid  and  the  surface  orientation,  Numb,  and  the  Lambertian  strength, 
A',  are  obtained.  If  not,  the  Lambertian  component  is  considered  not  to  exist. 

Step  2:  Specular  triangle  determination  We  subtract  the  base  value  and  the 
Lambertian  brightness  values,  extracted  in  the  former  step,  from  the  measured 
brightness.  We  obtain  the  specular  brightness  as  follows: 

V[  -  Ii  —  A' Si  •  Niamb  —  A'Euuni,  —  Edef  . 

Theoretically,  three  source  directions,  giving  three  largest  specular  brightness 
values,  form  the  specular  triangle.  However,  when  the  specular  source  direction 
of  a  surface  patch  agrees  with  the  light  source  direction,  the  second  and  the 
third  largest  direction  becomes  unstable,  and  does  not  always  belong  to  the  same 
triangle. 

In  order  to  compensate  this  instability,  we  use  the  following  procedure.  We 
select  the  two  largest  brightness  directions  among  the  specular  brightness,  (/•',  /  = 
1, . . . ,  36).  Usually,  the  two  light  sources  of  the  largest  values  are  adjacent  with 
each  other.  In  this  case,  the  common  edge  of  the  sources  belongs  to  two  triangles. 
We  will  choose  the  one  having  the  larger  brightness  at  the  third  vertex,  and  form 
the  specular  triangle. 

Sometimes,  the  two  light  sources  giving  maximum  values  are  not  adjacent 
with  each  v,:her.  In  this  case,  we  will  pick  the  largest  direction  and  search  the  next 
largest  direction  among  the  adjacent  directions  around  it.  We  will  form  a  common 
edge  by  the  largest  and  the  direction  found  in  the  previous  search.  Then,  we  will 
find  the  specular  triangle  among  two  triangles  which  shares  the  common  edge. 


Step  3:  Table  lookup  The  previous  procedure  gives  the  specular  triangle,  i.e. 
three  specular  brightness  values,  1^  '  l\J  l"iy  In  order  to  refer  to  the  lookup  table, 
we  will  normalize  the  specular  brightness  as 


k-  1 


/"  + 1"  +  /" 
'<1  *2 
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where  k  represents  the  total  mesh  number  of  an  axis  of  the  lookup  table. 

Using  the  normalized  brightness,  we  can  refer  to  the  lookup  table.  In  order  to 
obtain  accurate  orientation,  we  will  use  the  following  interpolation  method. 

The  first  step  generates  4  sets  of  indices  around  /■", 


( l,m,k —  1  —  /  —  m), 

(l+l,m,k  —  2  —  l  —  m), 

(l,m  +  l,k  —  2  —  /  —  m), 

(/+  l,m  +  \,k  —  3  —  /  —  m) 

where  /  =  [/•['] ,  m  =  [/■'']  and  /,  =  /•"  —  l ,  mr  =  -  m.  See  Figure  1 1. 

The  second  step  actually  refers  to  the  table  using  these  indices.  However,  since 
some  of  the  referred  cells  in  the  table  may  not  contain  orientation  information,  we 
will  execute  the  following  interpolation  procedure. 

•  Case  1:  When  three  or  four  cells,  that  are  pointed  to  by  the  indices,  contain 
orientations,  interpolate  between  them  by  using  the  residue,  ir ,  jr. 

•  Case  2:  When  one  or  two  cells,  pointed  to  by  the  indices,  contain  orienta¬ 
tions,  use  those  values  without  interpolation. 

•  Case  3:  When  none  of  the  cells  contains  an  orientation,  send  the  failure 
signal. 

Case  1  and  2  provide  surface  orientation,  Nspec.  By  using  this  orientation,  we 
calculate  the  specular  source  direction,  S„(N)  of  the  orientation.  Then,  we  find  the 
nearest  light  source  direction  5,  from  this  direction,  Sn(N).  Substituting  the  source 
and  specular  direction  in  Eq.  10  gives  the  theoretical  brightness.  The  specular 
albedo  B'  is  determined  as  the  ratio  of  the  measured  specular  and  the  theoretical 
brightness. 
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Figure  11:  Interpolation  method 
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4  Experimental  Results 

This  section  describes  experiments  conducted  to  evaluate  the  performance  of  the 
photosampling  method.  As  explained  previously,  the  method  can  extract  shape  and 
reflectance  of  Lambertian,  specular,  and  hybrid  surfaces.  In  order  to  evaluate  the 
accuracy  of  the  method,  first,  we  measure  three  kinds  of  spherical  surfaces,  known 
shape  and  known  reflectance:  Lambertian,  specular,  and  hybrid  surfaces.  Then, 
we  applied  the  method  to  other  interesting  surfaces  to  demonstrate  the  feasibility 
of  the  method  in  practical  applications  as  well  as  to  investigate  its  limitations. 

4.1  Specular  spherical  surface 

Figure  12  shows  a  0.5  inch  diameter  specular  ball  and  its  36  images  obtained  using 
the  photo-sampling  device.  By  applying  the  extraction  procedure  to  these  images, 
we  obtain  the  Lambertian  strength  as  shown  in  Figure  13(a)  and  the  specular 
strength  as  shown  in  Figure  13(b).  The  surface  orientation  distribution  is  obtained 
as  shown  in  Figure  14(a),  where  a  small  needle  depicts  the  orientation  at  that 
pixel.  The  detected  error  of  the  surface  normals  along  the  X  axis  is  shown  in  the 
Figure  14(c).  We  find  that  the  errors  are  less  than  1  degree  in  the  central  region  of 
the  ball. 

Figure  14(b)  shows  the  distribution  of  the  errors  in  computed  surface  orienta¬ 
tions.  The  figure  indicates  that  the  peripheral  area  has  larger  error  than  the  central 
area.  This  tendency  can  be  explained  by  the  discrepancy  of  the  actual  brightness 
distribution  of  a  light  source  from  the  theoretical  one.  Figure  15(a)  and  15(b) 
shows  the  fitting  results  of  the  25th  light,  which  is  located  in  the  horizontal  direc¬ 
tion,  and  the  36th  light,  which  has  the  maxmum  angle  with  respect  to  the  camera, 
as  the  examples  of  the  fitting  results  at  the  peripheral  area.  The  actual  brightness 
distribution  has  larger  discrepancy  from  the  theoretical  one  in  the  longitudinal 
direction  than  the  perpendicular  direction  as  shown  in  the  Figure  15. 

The  brightness  in  a  pixel  measured  by  a  TV  camera  is  the  resuJ  of  the  inte¬ 
gration  of  those  over  the  corresponding  area  on  the  diffuser.  On  the  other  hand, 
the  theoretical  brightness  is  defined  as  multiplication  of  the  pixel  area  with  those 
at  the  center  direction  of  the  pixel.  The  variance  in  surface  orientations  within  a 
pixel  becomes  larger  in  the  circumference  of  the  ball;  thus  the  variance  in  bright¬ 
ness  within  a  pixel  increases  in  the  circumference  of  the  ball.  A  larger  brightness 
variance  causes  more  departure  of  the  integrated  brightness  from  those  given  by 
multiplication  of  the  pixel  area  with  those  at  the  center  of  the  pixel.  This  departure 
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causes  a  relatively  larger  error  at  the  circumference.  If  the  tables  for  specular 
surfaces  are  made  directly  from  the  brightness  of  the  specular  ball  images,  the 
accuracy  will  be  improved. 

We  can  also  observe  the  systematic  error  distribution  over  the  ball  correspond¬ 
ing  to  the  tessellations.  The  accuracy  of  an  obtained  surface  orientation  is  related 
to  the  gradient  of  a  brightness  distribution  function  of  an  extended  light  source. 
A  steeper  portion  of  the  function  gives  a  more  accurate  orientation.  The  gradient 
of  the  source  distribution  function  is  not  constant  over  a  triangle.  It  becomes  rel¬ 
atively  flat  around  the  peripheral  area  of  a  triangle.  Consequently,  errors  increase 
in  that  region. 

Therefore,  it  is  desirable  for  the  function  to  have  a  steep  gradient  at  the 
peripheral  area  of  a  triangle.  In  order  to  achieve  this  condition,  we  should  have 
a  larger  a  than  the  width  of  a  triangle.  In  other  words,  the  distance  between 
the  light  sources  and  the  diffuser  should  be  wider  than  those  given  by  the  design 
specification  in  Eq.  8. 

4.2  Lambertian  spherical  surface 

Figure  16  shows  a  0.5  inch  diameter  matt-white  painted  Lambertian  ball  and 
its  36  images  taken  under  our  device.  Figure  17(a)  represents  the  Lambertian 
strength,  and  Figure  17(b)  represents  the  specular  strength.  Figure  18(a)  shows 
the  orientation  distribution  depicted  as  a  needle  map. 

Figure  18(b)  shows  the  error  distributions  over  the  sphere.  Figure  18(c)  shows 
the  error  distributions  along  x  axis.  The  average  error  is  2%  and  the  maxmum  error 
is  3.5%.  We  can  conclude  that  the  Lambertian  ball  has  larger  errors  in  obtained 
orientations.  This  is  because  the  cosine  curve,  the  photometric  function  of  a 
Lambertian  surface,  has  more  flat  gradient  than  that  for  a  pure  specular  surface. 

In  addition,  the  maxmum  brightness  of  Lambertian  surfaces  is  about  only 
20%  of  those  of  specular  surfaces.  Although  we  use  extended  light  sources  to 
detect  both  specular  and  Lambertian  surfaces  with  the  same  order  of  brightness,  to 
equate  the  maxmum  brightness  of  Lambertian  surface  to  those  of  specular  surfaces 
requires  that  the  termination  angle  be  larger  than  90°.  Thus,  we  cannot  realize 
such  large  termination  angles5. 

3 It  is  still  reasonable  to  make  Lambertian  surfaces  darker  than  specular  surfaces  because  the 
Lambertian  components  are  extracted  from  more  data  than  specular  components  by  using  the  least 
mean  square  fitting  method  which  is  stable  with  noise. 
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4.3  Hybrid  spherical  surface 

Examples  of  the  hybrid  surfaces  include  smooth  plastic  and  glossy  painted  sur¬ 
faces.  To  examine  the  system  performance  on  such  hybrid  surfaces,  we  use  a 
glossy  painted  ball  in  this  experiment.  Figure  19  shows  a  0.5  inch  diameter  glossy 
painted  ball  and  its  36  images.  Figure  20(a)  and  Figure  20(b)  represent  the  Lamber¬ 
tian  and  specular  strength,  respectively.  Figure  21(a)  shows  the  needle  map  of  the 
extracted  surface  normals.  F  ,;ure  21(b)  and  Figure  21(c)  depict  error  distributions 
in  surface  orientation  of  an  entire  sphere  and  along  the  x-axis,  respectively. 

4.4  Industrial  object  surface:  Dip  Switch 

A  dip  switch,  shown  in  Figure  22,  is  an  example  of  the  plastic  object  with  a 
complicated  shape.  Figure  23(a)  shows  the  Lambertian  strength.  Figure  23(b) 
shows  the  specular  strength.  As  we  expected,  the  system  successfully  obtained 
constant  Lambertian  strength  distribution  over  the  complicated  shape.  Although 
the  switch  has  two  different  albedo  areas,  the  distribution  of  the  surface  normals, 
depicted  as  the  needle  map  in  Figure  23(c),  is  seen  not  to  be  influenced  by  this 
albedo  variation.  However,  concave  and  side  areas  are  not  detected  correctly  due 
to  the  occlusion.  For  example,  the  small  circular  concave  on  the  switch  is  extracted 
with  smaller  albedo  than  actual,  while  the  adjacent  pan  to  the  switch  is  detected 
with  inaccurate  surface  normal. 

4.5  Solder  Joint 

Figure  24(a)  shows  a  solder  joint  Figure  24(b)  shows  the  needle  map  of  the 
surface  normals. 

4.6  Transparent  surface 

Transparent  objects  do  not  have  a  Lambertian  component,  but  do  have  a  specular 
component  As  an  example  of  a  transparent  surface,  we  have  used  the  surface  of 
a  plastic  lens  as  shown  in  Figure  25(a) 

The  plastic  lens  causes  double  reflections;  a  light  ray  hits  the  surface  from 
the  air  to  the  plastic  lens  and  a  penetrated  light  ray  hits  the  back  surface  from  the 
plastic  to  the  air.  Figure  25(b)  contains  such  double  reflections. 
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In  order  to  measure  the  only  front  reflection,  we  paint  the  lower  surface  with 
dull  black  spray.  Figure  26  shows  the  result.  Here,  the  length  of  each  needle  is 
multiplied  by  5  times.  The  errors  at  the  center  is  due  to  the  hole  of  TV  camera. 
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5  Conclusions 


This  paper  describes  a  three-dimensional  photometric  sampling  method  to  de¬ 
termine  shape  and  reflectance  of  glossy  surfaces  by  measuring  image  brightness 
under  various  light  source  directions  and  viewing  from  a  single  direction. 

Extended  light  sources  make  it  possible  to  ensure  the  detection  of  both  Lamber¬ 
tian  and  specular  reflections.  We  have  distributed  36  light  sources  on  the  vertices  of 
a  one  frequency  icosahedron  circumscribing  a  diffused  sphere  to  generate  evenly 
distributed  extended  light  sources. 

An  extraction  algorithm  uses  the  set  of  image  brightness  values  measured 
at  each  surface  point  in  order  to  compute  orientation  as  well  as  the  strengths 
of  the  Lambertian  and  specular  reflection  components.  In  order  to  increase  the 
accuracy  in  measured  orientations  and  reflectance  parameters,  we  have  developed 
a  calibration  method  to  overcome  the  the  difference  between  the  theoretical  and 
the  actual  distribution  of  extended  source  brightness. 

The  experimental  results  for  Lambertian  surfaces,  specular  surfaces,  and  hybrid 
surfaces  show  high  accuracy  in  both  orientations  and  reflectance  parameters.  We 
have  demonstrated  the  ability  of  our  system  to  detect  surface  orientations  and 
reflectance  parameter  variations  on  a  plastic  and  transparent  surfaces,  as  well. 
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(a) 


Figure  12:  A  specular  ball;  (a)  Picture  of  a  specular  ball;  (b)  36  images  of  the 
specular  ball  using  the  3D  photosampler. 
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(b) 

Figure  12  (cont) 
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Figure  13:  Strength  images;  (a)  Lambertian  strength  image  for  the  specular  ball; 
(b)  Specular  strength  image  for  the  specular  ball. 


Figure  14:  Needle  map  for  the  specular  ball;  (a)  Needle  map;  (b)  Needle  map  for 
the  orientation  error  of  the  specular  ball:  the  needle  size  is  enhanced  in  15  times; 
(c)  The  orientation  error  along  the  x  axis  of  the  specular  ball. 
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(b) 

Figure  15:  The  fitting  result;  (a)  The  extended  light  source  25;  (b)  The  extended 
light  source  36. 
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(a) 

Figure  16:  A  Lambertian  ball;  (a)  Picture  of  the  Lambertian  ball;  (b)  36  images  of 
a  Lambertian  ball  under  the  photosampler. 
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(b) 

Figure  16  (cont) 
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Figure  17:  Strength  images;  (a)  Lambertian  strength  image  for  the  Lambertian 
ball;  (b)  Specular  strength  image  for  the  Lambertian  ball. 


Figure  18:  Needle  map;  (a)  Needle  map  of  the  Lambertian  ball;  (b)  Orientation 
errors  of  the  Lambertian  ball.  Each  length  is  multiplied  by  15;  (c)  The  orientation 
error  along  the  x  axis  of  the  Lambertian  ball. 
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(a) 


Figure  19:  A  painted  ball  (hybrid  surface);  (a)  Picture  of  the  ball;  (b)  36  images 
of  the  ball. 
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Figure  20:  Strength  images;  (a)  Lambertian  strength  image  of  the  painted  ball;  (b) 
Specular  strength  image  of  the  painted  ball. 


Figure  21:  Needle  map  of  the  painted  ball;  (a)  Needle  map;  (b)  Error  distribution; 
(c)  The  orientation  error  along  the  x  axis  of  the  painted  ball. 
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Figure  22:  Picture  of  a  dip  switch. 
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(a) 


(b) 


(c) 


Figure  23:  Result  images:  (a)  Lambertian  strength  image  for  the  dip  switch;  (b) 
Specular  strength  image  for  the  dip  switch;  (c)  Needle  map  of  the  dip  switch. 
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(b) 


Figure  24:  A  solder  joint;  (a)  Picture  of  a  solder  joint;  (b)  Needle  map  for  the 
solder  joint. 
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(a) 


(b) 


Figure  25:  Plastic  lens:  (a)  Picture  of  a  plastic  lens;  (b)  two  reflections  are  observed 
on  the  plastic  lens. 


Figure  26:  Needle  map  of  the  plastic  lens. 


47 


References 


[1]  C.  Brown.  Fast  display  of  well-tessellated  surfaces.  Computer  and  Graphics , 
4(4):77-85,  April  1979. 

[2]  E.N.  Coleman  and  Jain.  R.  Obtaining  3-dimensional  shape  of  textured  and 
specular  surface  using  four-source  photometry.  Computer  Graphics  and 
Image  Processing,  18(4):309-328, 1982. 

[3]  G.  Healey  andT.O.  Binford.  Local  shape  from  specularity.  Computer  Vision, 
Graphics,  and  Image  Processing,  42:62-86, 1988. 

[4]  B.K.R  Horn.  Obaining  shape  from  shading.  In  P.H.  Winston,  editor.  The 
Psychology  of  Computer  Vision,  pages  115-155.  McGraw-Hill,  New  York, 
1975. 

[5]  K.  Ikeuchi.  Determining  surface  orientation  of  specular  surfaces  by  using 
the  photometric  stereo  method.  IEEE  Trans.  Pattern  Analysis  and  Machine 
Intelligence,  PAMI-3(6):661-669,  November  1981. 

[6]  K.  Ikeuchi.  Recognition  of  3-d  objects  using  the  extended  gaussian  image. 
In  International  Joint  Conf.  on  Artificial  Intelligence,  pages  595-600, 1981. 

[7]  K.  Ikeuchi  and  K.  Sato.  Determining  reflectance  parameters  using  range  and 
brightness  images.  In  Proc.  of  Intern.  Conf.  on  Computer  Vision,  Osaka, 
JAPAN,  December  1990.  a  longer  version  is  avaiable  as  CMU-CS-90-106. 

[8]  G.J.  Klinker,  S.A.  Shafer,  and  T.  Kanade.  The  measurement  of  highlights  in 
color  image.  Intern.  Journal  of  Computer  Vision,  2(1),  1988. 

[9]  S.K.  Nayar,  K.  Ikeuchi,  and  T.  Kanade.  Extracting  shape  and  reflectance  of 
lambertian,  specular,  and  hybrid  surfaces.  Technical  Report  CMU-RI-TR- 
88-14,  Carnegie  Mellon  University,  Robotics  Institute,  August  1988. 

[10]  S.K.  Nayar,  K.  Ikeuchi,  and  T.  Kanade.  Surface  reflection:  physical  and 
geometrical  perspectives.  Technical  Report  CMU-RI-TR-89-7,  Carnegie 
Mellon  University,  Robotics  Institute,  March  1989. 

[11]  A.P.  Pentland.  Local  shading  analysis.  IEEE  Trans  Pattern  Analysis  and 
Machine  Intelligence,  PAMI-6(2):  170-1 87,  March  1984. 


48 


[12]  A.C.  Sanderson,  L.E.  Weiss,  and  S.K.  Nayar.  Structured  highlight  inspec¬ 
tion  of  specular  surfaces.  IEEE  Trans,  on  Pattern  Analysis  and  Machine 
Intelligence,  10(l):44-55,  January  1988. 

[13]  MJ.  Wenninger.  Spherical  Models.  Cambridge  Univ.  Press,  Cambridge, 
UK,  1979. 

[14]  L.B.  Wolff.  Spectral  and  polarization  stereo  methods  using  a  single  light 
source.  In  Proc.  of  DARPA  Image  Understanding  Workshop,  pages  810-820. 
Science  Application  Inc.,  February  1987. 

[15]  L.B.  Wolff.  Using  polarization  to  separate  reflection  components.  In  Proc. 
of  IEEE  Conf  on  Computer  Vision  and  Pattern  Recognition ,  pages  363-369, 
1989. 

[16]  RJ.  Woodham.  Reflectance  map  techniques  for  analyzing  surface  defects 
in  metal  castings.  Technical  Report  AI-TR-457,  Massachusetts  Institute  of 
Technology,  Artificial  Intelligence  Laboratory,  Cambridge,  MA,  1978. 


49 


